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EXAMINER'S AMENDMENT 

An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided by 
37 CFR 1.312. To ensure consideration of such an amendment, it MUST be submitted no 
later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with Kevin Shao on 2/28/1 1. 

The application has been amended as follows: 

1. (Previously presented) A method comprising: 

extracting a first data from a display buffer, the first data being generated by a 
first application and being associated with a user interface from the first application; 
recognizing a layout from the first data; and 

using the layout to create an overlay to display a second data generated by a 
second application, wherein there is no direct link between the first application and the 
second application, and wherein the first data is extracted from the display buffer without 
cooperation of the first application at runtime. 

2. (Original) The method of claim 1, wherein recognizing the layout comprises 
performing a pattern recognition operation on the first data to create the layout. 

3. (Original) The method of claim 1, wherein using the layout to create the overlay 
comprises: 

determining an overlay location on the layout to place the second data based on 
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known information about the layout; 

generating the overlay of the layout; 
placing the second data in the overlay; and 
merging the overlay with the layout. 

4. (Original) The method of claim 3, wherein the overlay location has a context 
consistent with the second data. 

5. (Original) The method of claim 4, wherein the context is provided by the first 
application, and wherein a user interacts with the second application using the context. 

6. (Original) The method of claim 1, further comprising: 

writing the overlay in the display buffer such that the second data is displayed at 
the overlay location without changing sections of the first data outside of the overlay 
location; 

displaying information in the display buffer; and 

interacting with the second application through the second data at the overlay 
location. 

7. (Original) The method of claim 6, further comprising running the first application 
in the background while interacting with the second application. 

8. (Original) The method of claim 1, wherein the first application runs 
independently from the second application. 
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9. (Currently Amended) A machine-readable storage medium providing 
instructions, which when executed by a set of one or more processors, cause said set of 
processors to perform the following: 

extracting a first data from a display buffer, the first data being generated by a 
first application and being associated with a user interface from the first application; 
recognizing a layout from the first data; and 

using the layout to create an overlay to display a second data generated by a 
second application, wherein there is no direct link between the first application and the 
second application, and wherein the first data is extracted from the display buffer without 
cooperation of the first application at runtime. 

10. (Currently Amended) The machine-readable storage medium of claim 9, wherein 
recognizing the layout comprises performing a pattern recognition operation on the first 
data to create the layout. 

1 1 . (Currently Amended) The machine-readable storage medium of claim 9, wherein 
using the layout to create the overlay comprises: 

determining an overlay location on the layout to place the second data based on 
known information about the layout; 

generating the overlay of the layout; 
placing the second data in the overlay; and 
merging the overlay with the layout. 
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12. (Currently Amended) The machine-readable storage medium of claim 11, 
wherein the overlay location has a context consistent with the second data. 

13. (Currently Amended) The machine-readable storage medium of claim 12, 
wherein the context is provided by the first application, and wherein a user interacts with 
the second application using the context. 

14. (Currently Amended) The machine-readable storage medium of claim 9, further 
comprising: 

writing the overlay in the display buffer such that the second data is displayed at 
the overlay location without changing sections of the first data outside of the overlay 
location; 

displaying information in the display buffer; and 

interacting with the second application through the second data at the overlay 
location. 

15. (Currently Amended) The machine-readable storage medium of claim 14, further 
comprising running the first application in the background while interacting with the 
second application. 

16. (Currently Amended) The machine-readable storage medium of claim 9, wherein 
the first application runs independently from the second application. 
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17. (Previously presented) A computer system, comprising: 

a bus; 

a data storage device coupled to the bus; and 

a processor coupled to the data storage device, the processor operable to receive 
instructions which, when executed by the processor, cause the processor to perform a 
method comprising: 

extracting a first data from a display buffer, the first data being generated by a 
first application and being associated with a user interface from the first application; 

recognizing a layout from the first data; and 

using the layout to create an overlay to display a second data generated by a 
second application, wherein there is no direct link between the first application and the 
second application, and wherein the first data is extracted from the display buffer without 
cooperation of the first application at runtime. 

18. (Original) The system of claim 17, wherein recognizing the layout comprises 
performing a pattern recognition operation on the first data to create the layout. 

19. (Original) The system of claim 17, wherein using the layout to create the overlay 
comprises: 

determining an overlay location on the layout to place the second data based on 
known information about the layout; 

generating the overlay of the layout; 
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placing the second data in the overlay; and 

merging the overlay with the layout. 

20. (Original) The system of claim 19, wherein the overlay location has a context 
consistent with the second data. 

21 . (Original) The system of claim 20, wherein the context is provided by the first 
application, and wherein a user interacts with the second application using the context. 

22. (Original) The system of claim 17, further comprising: 

writing the overlay in the display buffer such that the second data is displayed at 
the overlay location without changing sections of the first data outside of the overlay 
location; 

displaying information in the display buffer; and 

interacting with the second application through the second data at the overlay 
location. 

23. (Original) The system of claim 22, further comprising running the first 
application in the background while interacting with the second application. 

24. (Original) The system of claim 17, wherein the first application runs 
independently from the second application. 
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25. (Previously presented) A method, comprising: 

modifying data in a display buffer that is generated by a first application with data 
generated by a second application without cooperation of the first application at runtime, 
the first application running independently from the second application; and 

receiving input in response to user interactions with the second application 
through a user interface associated with the data generated by the first application, 
wherein the data generated by the second application is placed in a location in the user 
interface, wherein the location is contextually consistent with the data generated by the 
second application. 

26. (Original) The method of claim 25, wherein modifying data in the display buffer 
comprises: 

performing a pattern recognition operation on the data generated by the first 
application to create a layout; and 

forming an overlay with the layout and with predetermined information about a 
display corresponding to the user interface, the overlay used to determine placement of 
the data generated by the second application in the display. 

27. (Original) The method of claim 26, wherein the layout comprises of grid cells 
corresponding to display areas in the user interface, and wherein the data generated by 
the second application is placed in the grid cells. 
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28. (Original) The method of claim 25, wherein the first application runs in the 
background while the user interacts with the second application. 

29. (Currently Amended) A machine-readable storage medium providing 
instructions, which when executed by a set of one or more processors, cause said set of 
processors to perform the following: 

modifying data in a display buffer that is generated by a first application with data 
generated by a second application without cooperation of the first application at runtime, 
the first application running independently from the second application; and 

receiving input in response to user interactions with the second application 
through a user interface associated with the data generated by the first application, 
wherein the data generated by the second application is placed in a location in the user 
interface, wherein the location is contextually consistent with the data generated by the 
second application. 

30. (Currently Amended) The machine-readable storage medium of claim 29, 
wherein modifying data in the display buffer comprises: 

performing a pattern recognition operation on the data generated by the first 
application to create a layout; and 

forming an overlay with the layout and with predetermined information about a 
display corresponding to the user interface, the overlay used to determine placement of 
the data generated by the second application in the display. 
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3 1 . (Currently Amended) The machine-readable storage medium of claim 30, 
wherein the layout comprises of grid cells corresponding to display areas in the user 
interface, and wherein the data generated by the second application is placed in the grid 
cells. 

32. (Currently Amended) The machine-readable storage medium of claim 29, 
wherein the first application runs in the background while the user interacts with the 
second application. 

33. (Previously presented) A computer system, comprising: 

a bus; 

a data storage device coupled to the bus; and 

a processor coupled to the data storage device, the processor operable to receive 
instructions which, when executed by the processor, cause the processor to perform a 
method comprising: 

modifying data in a display buffer that is generated by a first application with data 
generated by a second application without cooperation of the first application at runtime, 
the first application running independently from the second application; and 

receiving input in response to user interactions with the second application 
through a user interface associated with the data generated by the first application, 
wherein the data generated by the second application is placed in a location in the user 
interface, wherein the location is contextually consistent with the data generated by the 
second application. 
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34. (Original) The computer system of claim 33, wherein modifying data in the 
display buffer comprises: 

performing a pattern recognition operation on the data generated by the first 
application to create a layout; and 

forming an overlay with the layout and with predetermined information about a 
display corresponding to the user interface, the overlay used to determine placement of 
the data generated by the second application in the display. 

35. (Original) The computer system of claim 34, wherein the layout comprises of 
grid cells corresponding to display areas in the user interface, and wherein the data 
generated by the second application is placed in the grid cells. 

36. (Original) The computer system of claim 33, wherein the first application runs in 
the background while the user interacts with the second application. 

37. (Previously presented) A method comprising: 

reading raster data from a raster display buffer containing an image generated by 
a first application without cooperation of the first application at runtime; 

performing a pattern recognition on the image to generate a pattern; 

applying predetermined information about the image with the pattern to determine 
a layout of the image; 

generating an overlay using the layout of the image; and 
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placing data generated by a second application on the overlay. 

38. (Original) The method of claim 37, further comprising writing the overlay into 
the raster display buffer. 

39. (Original) The method of claim 37, wherein the image comprises a user interface 
from the first application, and wherein a user interacts with the second application 
through the user interface while the first application runs in the background. 

40. (Original) The method of claim 39, wherein while the user interacts with the 
second application, the first application has no control of input received from the user. 

Allowable Subject Matter 

The following is an examiner's statement of reasons for allowance: 1-40. 

Based on board's decision render on 2/25/11, the prior art either alone or 
in combination doesn't show or teach "extracting a first data from a display buffer, 
the first data being generated by a first application and being associated with a user 
interface from the first application; 

recognizing a layout from the first data; and 

using the layout to create an overlay to display a second data generated by a 
second application, wherein there is no direct link between the first application and the 
second application, and wherein the first data is extracted from the display buffer without 
cooperation of the first application at runtime" in combination with other features. 
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Any comments considered necessary by applicant must be submitted no later than 
the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to SIMON KE whose telephone number is (571)272-4062. 
The examiner can normally be reached on M-Th and Alternate Fridays 8:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessfiil, the examiner's 
supervisor, Dennis Chow can be reached on (571) 272-7767. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
Customer Service Representative or access to the automated information system, call 
800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Peng Ke 
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